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fit  AOSTR  AC  T 


A  y*rl*ia.  ort.r  fllttr  li  charactarixed  by  a  filter  »uch  that  the  dlaenelon  of  the  atate  vector 
vart..  vlth  time.  A  .pedal  important  ca.e  occur,  for  Unear  sy.tea.  which  can  be  generated  from  aam- 
pUng  a  unear  dytmmic  .y.t«»,  x  -  A(t)x  .  b(t),  y  «  M(t)x  *  Tl(t)  where  the  tem  b(t)  =  5:  b^  6^(t  - 
and  6,(-r)  1.  the  a.y»etric  Dirac  function  and  the  Tj  are  known  a.  knot,.  Recursive  algorithm,  are 
obtalled  given  that  are  hnown.  Because  of  the  simplicity  of  the  algorithms,  it  is  feasible  to 
..tlrnate  both  the  magnitude  of  b^  and  the  knot  position  Tj  by  appending  a  detector  to  process  the 
redduds  or  the  innovation  proces..  Using  cla..ic  hypothesis  theory  we  then  adaotlvely  change 
the  model  by  assigning  a  value  of  b^  0  and  provided  data  is  observ'ed  for  t  >  T^. 

Thus,  the  variable  order  filter  a,  appUed  to  linear  dynamic  ayatems  car.  be  thought  of  as  a 
curvcfltting  approach  to  the  solution  of  the  protl«s  of  divergence  of  recursive  filters  due  to  model 
inaccuracies.  This  is  accompU.hed  In  a  method  directly  amtlogous  to  increasing  the  degree  of  oolynom- 
id  in  a  curve  fit  to  rtduce  the  truncation  error.  The  curve  fitting  carr  be  done  adar.tively  reducing 
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1. 


INTRODUCTION 


Polynomial  spline  functions  are  a  class  of  piecewise  continuous 
polynomial  functions  satisfying  continuity  conditions  at  certain  joint 
points,  known  as  knots.  They  are  a  generalization  of  polynomials  and  have 
been  found  to  have  highly  desirable  characteristics  as  approximating,  in¬ 
terpolating  and  curve-fitting  functions.  The  latter  property  is  of  special 
Interest  in  application  to  the  problem  of  real-time  estimation  of  state 
vectors  of  linear  dynamic  systems  subject  to  intermittent  discontinuous 
changes.  Briefly  stated,  a  pol3rnomial  spline  function  of  order  M  is  the 
order  Integral  of  a  step  function.  The  points  of  discontinuity  of  the 
step  function  are  the  knots. 

The  theory  of  variable  order  filters  was  derived  from  the  author's 

previous  studies on  the  application  of  polynomial  splines  and 

fl  21* 

generalized  splines^  *  *  to  statistical  filter  theory.  The  current 
algorithms  for  the  variable  order  filters  are  applications  of  and  exten¬ 
sions  of  the  spline  function  concepts.  This  class  of  filters  will  have 
application  to  smoothing  and  prediction  of  sampled  data  systems  for  the 
following  classes  of  state  estimation  problems: 

(a)  Estimation  of  the  state  vector  of  a  linear  dynamic  system  with 
rapidly  changing  trajectories  as,  for  example,  a  maneuvering 
vehicle; 

(b)  High  precision  estimation  of  the  state  vector  based  on  long 
observational  intervals  where  the  estimation  accuracy  may  be 
limited  by  modelling  errors; 

(c)  Estimation  of  the  state  vector  when  certain  components  of  the 
state  vector  are  subject  to  Intermittent  discontinuous  changes 
as,  for  example,  staging  rockets. 


Generalized  splines  are  solutions  to  certain  classes  of  homogeneous 
linear  differential  equations  with  suitably  restricted  time  varying  co¬ 
efficients  so  as  to  ensure  that  the  adjoint  equation  exists 


1 


Spline  functions  have  been  studied  extensively,  and  two  excellent 

[1  2] 

books  ’  present  a  comprehensive  review  of  the  applications  and  theory 
of  spline  functions. 

[3  4] 

C.  deBoor  and  J.  R.  Rice  ’  at  Purdue  University  have  programmed 
least-square  cubic  spline  approximations  for  fixed-knot  and  variable-knot 
cases.  In  the  fixed-knot  case,  the  positions  of  the  knots  are  fixed 
a  priori;  in  the  variable-knot  case,  the  knot  location  is  adjusted  to  the 
data  set  to  minimize  the  mean-square  error  between  the  spline  curve  and 
the  data  set. 

The  property  of  splines  which  is  of  interest  in  this  application  is 
tl'at  for  the  same  number  of  free  parameters,  the  splines  are  able  to  more 
closely  approximate  a  large  class  of  functions  than  a  polynomial  curve  fit. 
^or  example,  if  a  cubic  spline  is  used  with  two  interior  joint  points,  then 
each  polynomial  section  has  four  coefficients  to  define  the  polynomial  for 
a  total  of  12  coefficients.  Typically,  in  the  spline  application,  one. 
requires  the  0*"^  to  2nd  derivative  to  be  continuous  at  each  interior  knot 
point.  This  induces  six  linear  equations  of  constraint  on  the  coefficients, 
leaving  six  free  parameters  to  be  deterained  by  least-squares  turve  fitting 
to  the  data  set.  The  experience  of  previous  investigators  is  that  for  some 
rather  rapidly  changing  functions,  such  a  spline  function  will  follow  the 
function  more  closely  than,  for  example,  a  least-square  polynomial  of  degree 
five,  which  also  has  six  free  parameters. 

Thi.'i  has  immediate  implications  for  tracking,  fur  example,  maneuver¬ 
ing  targets,  since  spline  functions  may  be  superior  to  polynomials  as  the 
baste  aijpruximat lull  lo  the  trait'cioiv. 


Tracking  is  essentially  a  real-time  process,  and  the  work  of  deBoor 
&  Rica  is  essentially  a  post-flight  analysis  tool.  That  is,  the  algorithms 
are  not  suitable  for  real-time  application.  Real-time  tracking  using  spline 
approximations  was  studed  by  H.  Schneider  and  G.  S.  Gorden  at  MIT,  1966  - 
1969^^'^^.  In  this  application,  Schneider  considers  the  approximation  of 
long  segments  of  a  ballistic  reentry  trajectory  by  a  sequence  of  M  -degree- 
polynomial  splines  with  equally  spaced  knots.  The  first  (H-1)  derivatives 
were  continuous  at  the  interior  knots.  Schneider  derives  recursive  algorithms 
in  which  the  free  parameters  defining  the  splines  are  updated  upon  receiving 
a  new  observation  in  an  optimal  manner.  That  is,  storing  only  the  present 
observation  and  the  previous  estimate  of  the  free  parameter  vector,  Schneider 
obtains  a  minimum-variance  estimate  o£  the  free  parameters  which  is  statls- 
ically  equivalent  to  having  processed  all  the  observations  in  a  batch  process. 
Schneider  also  clair^  that  the  splines  give  a  better  fit  (paaller  bias  error) 
than  pol'/nomials  for  the  same  number  of  parameters. 

In  Reference  (7)  "lecursive  Algorithms  for  Spline  Filters,"  the  author 
greatly  expanded  the  mo. lei  of  the  spline  filter  concept,  to  Include  linear 
combinations  of  known  functions  between  arbitrarily  spaced  knots,  and  con¬ 
straint  conditions  conaiEtlug  of  general  linear  operators  evaluated  at  the 
knots.  These  generaliaitlcns  are  highly  useful  for  the  tracking  of  the  class 
of  trajectories  of  interest. 

If  dynamic  equatioi,^  of  motion  are  available  for  the  trajectory,  one 
may  linearize  these  equations  about  some  nominal  parameters  which  determine 
Che  trajectory.  Then  the  gene'-alized  spline  filters  can  be  thought  of  as  a 


generalization  of  the  Kalman  filters  as  applied  to  piecewise-contlnuous 
functions,  with  additional  linear  constraints  at  the  knot  points.  In  this 
form,  the  -'daptive  techniques  for  placing  knots  may  be  an  attractive  method 
of  avoiding  some  of  the  difficulties  encountered  in  applying  Kalman  filters, 
especially  divergence  of  the  filter  from  the  data.  A  special  class  of  linear 
dynamic  equations  of  motion  with  impulsive  driving  functions  as  the  inputs 
at  the  knots  form  the  basis  for  th®  variable  order  filter  algorithms  developed 
in  Section  2. 

In  thle  paper,  the  concept  of  "variable  order  filters"  has  been  applied 
to  p  Liajactory  given  by  a  linear  dynamic  model  with  an  impulsive  driving 
funct-*  ^n.  The  estimate  of  the  state  vector  is  obtained  as  a  set  of  recursive 
equations  as  shown  in  Section  2.  The  time  at  which  the  impulsive  driving 
function  changes  plays  the  role  of  the  knots.  The  magnitude  of  the  vector 
coefficients  of  the  driving  function  are  the  unknowns  to  be  estimated.  In  the 
non-adaptive  mode  the  position  of  the  knots  are  assumed,  a  priori.  In  the 
adaptive  mode,  the  position  of  the  knot  is  to  be  determined  from  the  obser¬ 
vations  in  real  time. 

The  properties  of  the  optimum  variable  order  filter  are: 

A.  It  has  a  proving  memory  and  uses  all  the  data  from  the  time 
origin  to  the  present  to  obtain  a  minimum  variance  estimate  of 
Che  state  vector. 

B,  The  state  vector  changes  in  dimension  each  time  a  knot  Is  cfu-ased 
when  cib'jcrving  datu  in  real  time. 


C.  For  sufficiently  slow  increase  in  bias  error,  there  are  possi¬ 
bilities  for  obtaining  asymptotically  consistent  estimates  of  the 

state  vectcr;  that  is,  tha  total  erroi*  including  bias  is  driven 
to  zero. 

There  is  a  trade-off  in  mean-square  error  for  the  variable  order 
filter  between  the  increase  In  dimensions  of  the  state  vector  which  in¬ 
creases  the  noise  componenv  of  total  error  and  the  decrease  in  the  bias 
error  component.  If  the  noise  error  increases  too  rapidly,  it  may  not  be 
desirable  to  increase  the  dimension  of  the  state  vector.  An  algorithm  for 
collapsing  to  the  dimension  necessary  to  specify  the  trajectory  between 
knots  is  shown  also  in  Section  2. 

The  main  computational  penalty  in  using  the  optimum  variable  order 

filter  seems  to  be  storage.  If  the  state  vector  grows  to  dimension  ({>  the 

2 

storage  of  the  covariance  matrix  of  the  state  vector  is  of  the  order  /2, 

In  Section  2  the  algorithms  for  the  variable  order  filters  are  derived 
for  linear  dynamic  sy 'items  and  presented  in  detail.  This  is  the  non- 
adaptlve  mode.  In  Appendix  A  a  discussion  of  the  adaptive  version  is  given. 

In  Appendix  A  the  algorithms  developed  in  Section  1  are  applied  to 
the  specific  case  of  cubic  splines.  The  polynomial  splines  and  generalized 
epllnes  form  a  subset  of  the  class  of  f'lnctions  which  can  be  used  to  define 
the  linear  dynamic  systems  considered  in  this  paper. 


2.  VARIABLE  ORDER  FILTERS  FOR  LINEAR  DYNAHIC  SYSTEMS 


Consider  the  following  linear  dynamic  system  .^ef.  8): 


A(t)x  +  f  (t)  tel  S  [o.  T) 


y  -  M(t)x  +  n(t) 


where  A(t)  »  Is  an  n  x  n  matrix  of  functions*  each  of  which  Is 

measurable  on  I  and  (A(t)|  •  m(t)*  tel*  where  m(t)  Is  Lebesgue-lntegrable 
on  I.  Similar  properties  hold  for  the  elements  of  nxl  column  matrix 
f(t)  and  the  rxn  matrix  H(t).  Then  It  Is  shown  In  Ref.  9  that  unique 
solutions  exist  for  Eq.  1. 

The  solution  can  be  written  in  terms  of  the  initial  state  vector 


x(o)  “X  as 
o 


x(t)  -  (t*  o)x  +  J  w.  (t,  X)  f(X)  d  t  >  o  (3) 

+  o  ” 


Note  that  w.(t,  X)  is  the  n  x  n  state  transition  matrix  determined  for 
t  >  X  as  a  solution  of 


dw^(t,  X) 
dt 


“  A(t)w^(t,  \) 


t  >  X  (4) 


6 


with  the  initial  condition 


w  (X,X)  «  I 
+  n 


(5) 


where  is  u  x  u  identity  matrix. 

Note  that  w^(t,  X)  is  the  response  to  a  set  of  asymmetrical  unit 
Impulses  fjj^(t)  “  6^(t  -  X)  where  the  unit  impulse  6^(t  -  X)  is  defined  by 


b 

I 


a+o 


f(e)  6^(5  -  x)  d? 


0  if  X  <  a  or  ^  b 
f(x  -f  0)  if  a  <  X  <  b 


a  <  b  (6) 


the  asymmetrical  unit  step  function  is  given  by 


B+(x) 


0 

1 


for  X  ^  0 
for  X  >  0 


(7) 


In  £q.  2,  y  is  an  r  X  1  observation  vector  M(t)  la  an  n  x  r  observation 
system  transformation  with  continuous  elements  M^j(t)  and  nCt)  is  an 
r  X  1  vector  representing  white  observation  noise.  The  relationship 
between  the  observati'on  and  the  initial  state  vector  can  be  written  by 
substituting  Eq.  3  into  Eq.  2  as 


y  =  M(t)w^(t,o)x^  4  M(t)  I  w^(t,  X)f(X)  dX  +  n(t) 

o 


t  >  o 


(8) 
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and  is  of  dimension  n.  Let  o  £  t  <.  then 


y  +  M(t)  [w^(t,  o)x^  +  w^(t,  +  Tj(t) 


Define  an  augmented  state  vector  as 


Ij  >  t  >  o 


with  transition  matrix  ^  (t,  o)  given  by 


Therefore 


“  w^(t,  o)  w^(t,  T^) 


o)  ,W’(0) 


The  state  vector  has  increased  in  dimensions  from  n  to  2n.  In  general 

it  is  not  desirable  to  Increase  the  state  vector  dimensions  any  more  than 

necessary.  Let  it  be  known  a  priori  that  only  some  subset  of  components 

of  the  n  X  1  vectocs  will  differ  from  zero,  bsually  this  will  be  the 

* 

j.ist  component  b  lii  this  case,  if  we  let  represent  the  x  1 


vector  of  nonzero  components  (rows)  and  w^(t,  T^)  the  corresponding  columns 

of  w^(t,  T  ).  Then  we  may  write  Eq.  16  in  terms  of  the  nonzero  components 

*  [ol 

of  bj  and  the  state  vector  x  (t)  as 


or 


X^°^(t) 

* 

— 

aw 

x'H 

(t)  ■ 

w^(t,  t)  w^(t,  Tj^) 


U^(t  -  T^)I^ 


(1) 


X  "(t) 


U3, 


,111, 


(18) 


(19) 


Note  the  dropping  of  the  '  notation  for  x^^^(t),  w_j^^  ^(t,  x)  Indicates 
that  b*  is  being  used. 

Let  y(t)  be  obse:*ved  over  the  discrete  set  of  times  (t^,  t2,  t^  .  •  • 

The  following  section  presents  the  recursive  algorithms  for  the  minimum 

variance  estimates  of  the  state  vector  at  the  current  time  t,  and  the 

k+1 

equations  for  the  covariance  matrix  of  the  state  vector  estimation.  The 

minimum  variance  estimate  corresponds  to  a  growing  memory  filter  operating 

on  all  the  data  from  the  first  measurement  t,  to  the  current  time  t,  . , .  The 

1  k+1 

state  vector  grows  in  dlr.’nsion  by  q(u)  at  the  beginning  of  the  U  stage 
or  when  data  at  time  t  >  T^  (the  knot)  is  acquired.  In  Section  3*2  the 
algorithms  for  a  constant  order  filter  are  described-  In  this  filter,  the 
state  Vi.^K.ox  is  updated  at  the  oeginning  of  tne  stage,  but  maintained 


10 


at  the  dimension  n.  This  algorithm  Is  similar  to  a  constant  memory  filter 
and  is  not  optimal  in  a  minimum  variance  sense. 

The  constant  order  filter,  however,  requires  less  storage  and  under 
some  clrcinutaness,  any  he  more  desirable  if  the  noise  error  due  to  Increased 
dimension  of  the  state  vector  is  growing  more  rapidly  than  the  bias  or 
dynamic  modelling  error. 

In  the  algorithms  of  Sections  3.1  and  3.2  .  it  Is  assumed  that  the 
positions  of  the  knots  arc  known  a  priori.  Possible  mrdiflcatlons  of  the 
algorithms  to  make  them  adaptive,  l.e.,  to  determine  T^  from  the  data,  are 
discussed  In  Appendix  B. 


3.  RECURSIVE  ALGORITHMS  FOR  VARIABLE  ORDER  FILTER 


In  general,  we  can  write  the  following  recursive  relationship  for 

t, 

the  state  vectors,  transition  matrices  and  observation  siatrix  as  a  measure¬ 
ment  in  a  new  stage  is  introduced.  Define  for  U  »  0: 

i  ’ 


x^°^(t)  A  x(t) 


T,  >t>T>o=T 
1  —  —  —  o 


w^^°^ (t,  t)  ^  t) 


(20) 


M^°^(t)  ^  M(t) 


* 

b  -  0 
0 

Define  for  U  ®  0,  1,  2,  ...  N  -  1,  and  for  =  t  >  Ty 


^[U.l] 


(t) 


(21) 


(t.x)  . 

W^(t,  Ty^^) 

U 

J:  q(J) 
j=i 

(22) 

0 

0 

^+^^~'^U+l)^q(U+l) 

] 

(rn) 

ii? 


Let  y(t)  be  observed  over  a  discrete  set  of  observation  tines  {t^»t2****tj 


a.  be  the  ndLnlmum  variance  state  estinat  at  t^^ 
given  all  the  observations  up  to  and  Including  t^. 

b.  be  th«i  covariance  matrix  of  the  error  in 
x^®^ (k|k). 

c.  wjy^(k  -t-  l^k)  be  the  state  transition  natrlx  from 

d.  x^^^(k  +  l|k)  be  the  mininum  variance  estimate  of  the  pre¬ 
dicted  state  vector  at  time  t^^  based  on  the  measurements 
\:{y(tj^)»y(t2)  ••• 

e.  P^'^^Ck  +  l|k)  be  the  covariance  matrix  of  the  error  in 
x^^^  (k  +  l|k). 

f .  (k  +  1)  be  the  augmented  measurement  matrix  at  t^. 

g.  R(k  +  1)  be  the  measurement  noise  covariance  matrix 
associated  with  T)(t)  at  t^^. 

h.  K^^^(k  4  1)  be  the  filter  gain  at  tj^^^  and  be  the 
measurement  or  observation  at  t,  . , . 


Assume  that  the  initial  conditions  x  are  random  variables  with 

o 

_  * 
ensemble  mean  x  and  covariance  matrix  P  knovn  a  priori.  For  the  b. 

o  o  j 

assume  a  priori  estimates  of  b|  and  its  corresponding  covariance  matrices 
are  known.  Let  x^,  b*  be  Independent  of  each  other  and  the 
measurement  noise  as  well  as  the  position  of  the  knots  Tj^,  ... 

Then  one  may  write  the  following  algorithm  (Ref.  10,  page  270). 


3.1  ALGORITHM  A 

Within  the  stage  for  saiaples  2l  \  ^ 

Suppose;  tjj  >  T^,  5  T^;  U  »  1,  ...  N  -  1 

Initialize  the  stage  by  defining  the  a  priori  state  vector: 

-  l|k  -  1) 

IV  ;^"j(k-i|k-i)^ 

The  a  priori  covariance  vector  is  given  by 

2'.  P^^^(k  -  l|k  -  1)  - 

Form  the  augmented  transition  matrix: 


l4 


Fotm  the  augmented  observation  matrix: 

4*.  -  [M^^’^^(tj^)jOJ 

Compute  the  predicted  state  vector: 

5'.  x^®^[kjk  -  1]  -  wj^^Ik,  k  -  1]  x^^^k  -  ijk  -  1] 

Compute  the  predicted  error  covariance: 

6.’.  P^^^[k|k  -  1]  «  w|^^[k,  k  -  11  P^^hk  -  l|k  -  1]  w|^^[k,  k  -  l]’’^ 
Compute  the  filter  gdin  matrix: 

7  '.  K^^^(k)  -  P^“^[k|k-llM^^^(k)'^[M^^^(k)P^®^(k|k-l)M^^^(k)’^+R(k)r^ 
Process  the  observations  to  update  state  vector 

8'.  x^^^Ckjk)  -  x^^^(klk-l)  +  K^”^(k)Iyj^-M^”^(k)x^^^[klk-l]l 

Compute  the  covariance  matrix 

9’.  P^^^(klk)  -  M^^^(k)  P^^^(klk-l)  X 

m 

Iq(u)  ‘  ^  ■*■ 

. 

K*-^^(k)R(k) 

where 

U 

Q<U)  “  n  +  I  q(j) 
j=l 

Q(o)  =“  n 

tVl 

Q(U)  is  the  dimension  of  the  state  vector  in  the  U  stage. 


Test: 


10’.  yes  set  k  *  k  +  1,  and  retuxn  i.u  Eq.  3. 

If  no,  set  U  »  U  +  1  and  suppose  J(U)  Is  the  smallest  integer  for  vhlch 

*^k+j<U)  ^  ’^IH'l  »  ^lcfj(0)-l 

Set  k  +  j(U)  for  k  in  Eq.  l'  through  Eq.  10*  and  start  at  Eq.  1'. 

3.1.1  Initiation  Procedure,  Zero  Stage 

For  U  »  0  Eq.  1  through  Eq.  10,  reduce  to  the  standard  linear  Kalman 
filter  (Ref.  10,  page  270)  In  which  the  state  vector  transition  matrix  and 
observation  matrix  are  given  by  Eq.  20  valid  for  ^  t  ^  0.  Eq.  5'  become ‘S 
at  tj^,  that  is,  k  »  1, 


x^°^[l|o]  -  w^[l,olx^ 

P^°^[o|o]  - 

Equations  6', through  10'  are  then  obtained  by  obvious  substitution.  More 
generally,  when  one  wishes  to  change  from  a  variable  order  to  a  fixed  order 

filter  of  dimension  of  the  original  state  vector,  one  may  derive  initial 

* 

state  vectors  and  covariance  matrices  at  time  tj^_j^  noted  by  x(k  -  l|k  -  1), 
P(k  >  l|k  -  1)  and  substitute  M(k)  for  M^“^(k),  w^^^(k,  k  -  1)  for 
w^^^(k,  !  -  1)  in  Eqs.  5*  through  9*.  The  presence  of  the  _  then  indicic 
a  constant  order  filter. 
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3.2  ALGORITHM  B.  ALGORITHM  TO  CONVERT  TO  CONSTAKT  ORDER  FILTER 

Suppose  that  at  observation  in  the  stage »  it  Is  decided  to 
switch  from  a  growing  order  filter  to  a  constant  order  filter.  Then  assume 
one  lias  computed  from  the  observation  at  tj^^^  x^®^(k-2|k-2)  and 

P^®^(lt-2jk-2).  Let 

3i*  w^°^(k-l,k-2)  -  first  n  rows  of  w|®^ (k-l,k-2) 

1.  and  x^^^(k**l|k-2)  «  first  n  eloaents  of  the  state  vector 

i^®^(k-llk-2) 

Then  it  follows  thst: 

5. *  x^^^(k-l|k-2)  -  w^®^(k-l,k-2)  x^”^(k-2,k-2) 
and  the  covariance  of  x^®^ (k-l|  k-2)  la  given  by 

6. *  P(k-l|k-2)  -  w^”^(k-l»k-2)P^“^(k-2|k-2)w^^^(k-l,k-2)’*^ 

The  gain  matrix  is  given  by  Eq.  7^. with 

7. *  K(k-l)  -P((k-l|k-2))M(K-l)^  [M(k-l)P(k-l|k-2)M(k-l)^+R(k-l)]"^ 

The  updated  state  vector  and  covariance  matrix  after  processing  the  obser¬ 
vation  at  la  given  by 

9.*  P(k-l{k-l)  =  [I  -K(k-l)M(k-l)]P(k-l  k-2) [I  ~K(k-l)M(k-l)]'^ 

+  K(k-l)R(k)K(k-l)^  . 

The  starred  equations  are  parallel  to  the  same  equations  in  Algorithm  A, 
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8 .  *  x(k-l I  k-1)  -  X ^  (k-1 1  k-2)+K(k-l)  [y^_3^-M(k-l)x^®^  [k-l j  k-2 J  ] 

Equations  8*  and  9*^  of  Algorithm  B  derive  a  state  vector  of  dimensions  n  and 
its  covariance  matrix  which  contain  the  effects  of  each  of  the  impulse 
changes  to  the  state  vector. 

3.2,1  Approximately  Constant  Order  Filter 

Suppose  a  new  knot  appears  and  we  do  not  wish  to  increase  the  dimen-* 

slonallty  of  the  state  vector.  We  can  iticrease  the  dimensionality  vemporarlly 

for  a  set  of  measurement  to  obtain  an  estimate  of  b*  and  then  ''ollapse  the 

a 

order  using  Algorithm  B.  Let  us  demonstrate  this  for  a  spe'  Ific  cxampl.?  as 
follows:  One  starts  in  the  zero  stage  with  a  state  vector  of  dimension  n. 

Suppose  at  each  knot,  the  impulse  input  h^  has  only  the  Iasi  comyonen';,  as 

—  2  i'b 

a  nonzero  value  with  mean  b.  and  variance  a..  In  fno  stage  ore.  has 

ip  1 

a  state  vector  of  dimension  r.  for  the  k-1  obser.'-itton.  Note  tj^_.  £ 

t  >  T  ,  and  U  “  0,  is  used  in  .Ugorithm  A.  Foe  the  next  j-measurementa  one 
k  1 

uses  a  state  vector  of  dimension  n  +  1  applying  Algorithm  A  for  U  »  1  with 

*^fc+j-l  -  ‘^2*  ’^k+j  ^2*  *^k+i-l  \-l  Algorithm  B,  we 

obtain  "initial"  conditions  at  time  that  is,  the  n  dimension  state 

vector  {x(k+j-llk+j-l)},  and  its  (n  x  n)  covariance  matrix  P(k+j-l|k+j-l) . 

We  may  now  return  to  Algorithm  A  and  augment  x(k+1-l|kfj-l)  by  adding  b2jj, 
in  Eq.  1*  and  augment  to  the  a  priori  covariance  matrix  in  Eq.  2 

The  effect  of  collapsing  the  dimension  of  the  state  vector  is  to  obtain 
a  subopt imal  estimate  of  the  state  vector  in  the  sense  that  it  is  not  a 
minimum  variance  estimator.  However,  this  may  be  desirable  to  control  the 
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f;Qtal.  «rror  or  divergence  of  the  nodel  froei  tlie  data,  and  reduce  storage 
requireaHtnte . 
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APPENDIX  A 


By  the  Taylors  exfanslon  of  (lA)  about  t  one  can  easily  show  that 
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and  the  .knots 


Let  the  samples  occur  at  Integral  values  of  t  »  1,2,  ... 
occur  at  Tj  ®»  N  j .  j  .=  1,  2,  3,  with  N  *•  10. 

Further  let  the  Input  driving  function  £(t)  be  given  by 


e» 

f(t)  -  I  b,  6.(t-T,)  .  (4A) 

j-1  J  ^ 

I 


Where  the  elements  of  the  nxl  vector  b^ 
element  b  .  and 


are  known  to  be  zero  except  for  the 

I 


•  1 

E  {b  -  0 


Var  {b  "  0 
nj 


2 

J 


The  a  priori  values  of  E{x^}  -  x^  ■  0  and  Var  {x^}  -  are  known.  The 

random  variables  b  x  and  the  measurement  error  are  mutually  Independent, 
nj  o 

The  observations  are  given  by  the  scalar 


y(t)  “  x(t)  +  n(t) 

! 

where 

E{fi(t)}  -  0,  E{n(k)n(j)}  - 


j 


k,  otherwise  0. 


(6A) 


(7A) 
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That  is  M(t)  »  [1,0,0,  ...  0].  The  recursive  algorithm  A  is  used  for 
t  =  1,  2,  ...  N 


x'^°^[kjk-i]  -  w|°^  rk,k-i]  x^°hk-i|k-i] 


(8A) 


where 


i^°^[0j01  -  X  “  0 

*  '  o 


P^°^[klk-1]  -  w|°^[k,k-i:  P^°^[k-llk-ll  w|°^[k,k-l]'’^  (9A) 


with 


[k-llk-i]  =  p^,  k-i 


(lOA) 


K^°^(k) 


P^°^[k|k-1]  e 


4: 


p{°^  [kik-ll  K'^(k) 


1-1 


11 


ij  ^ 


where 


e^  =  [0,  0,  ...  1,  0,  ..  0]  is  the  Ixn  unit  vector  with  zeros 
for  each  component  except  the  j which  is  1.  The  double  subscript  on  P 
indicates  the  element  of  the  matrix.  The  optimum  estimate  of  the  current 
value  of  the  state  vector  is  then 


x^°^[k|k]  =  x^^'^Lklk-l]  +  Kj°^k)[>Y  -  x[°^[k!k-ll] 


(liA) 


with  covariance  matrix  [see  alternate  form  page  270  Ref.  10]. 


P^°\klk)  -  [I  -  K^°^(k)  c"]  P^°^(kik-13 

tl  X 


(12A) 


I 


when  k  »  10,  and  impulse  input  occurs  which  effects  the  filter  at  k  »  11. 
We  now  set  U  «  1  and  switch  back  to  Eq.  1'  of  Algorithm  A.  Setting  k  >■  11 
for  the  initial  condition  of  D  “  1  the  a  priori  state  vector  is  given  by 


*^’^'(10,10)  -  *^‘’^(10,101 


‘in  ■  “  -* 


The  a  priori  covariance  matrix  is  given  by 


P^^^CIO  10)  »  P^°M10|10] 


(13A) 


(lAA) 


The  augmented  transition  matrix  is  given  by 


w /^^(k,k-l)  = 


1  (k~10)' 


w^(k,k-l)  from  Eq.  3  4- 


(15A) 


(k-10)^ 


U_^(k-10) 


U^(k-lO) 


0 


for 


k  -  11,12,... 


where  (U)  =0  U  <  0 

*T* 

=•1  U  >  0 

(16A) 

(U)^  -  ,  U  >  0 

-  0  U  <  0 

The  recursive  fonsulatlon  given  by  Eq.  5* to  10' are  then  followed  with 
obvious  modifications  as  shown  la  equations  A-8  to  A-12  until  k  *  20.  At 
k  •»  20,  a  second  impulse  enters  which  requires  changing  U  from  1  to  2  and 
looping  back  to  equation  1’  of  algorithm  A  as  follows: 


^^^^20120] 


W 


[20l20] 


"20=-° 


P^^'^(20l20]  = 

P^^^[20l20] 

0 

0 

k 

2 

“aj 

(17A) 


(18  A) 
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M(k)  »  (209.) 

i. 


for  k  =  21,  22,  ... 

Algorithm  A  is  now  used  with  obvious  substitutions  for  k»21,  ...  30 
EXAMPLE  OF  CONSTANT  ORDER  FILTER 

Let  us  use  the  sample  example  of  the  cubic  splines  to  demonstrate 
the  constant  order  filter  algorithm  B.  In  the  zero^^  stage  one  has  obtained 


the  optimal  estimate  of  the  1x4  rtate  vector  r.  To  accommodate  the  family 
of  impulse  responses  consisting  of  1  ncn-zero  component,  the  state  vector 
is  augmented  as  given  by  Eq.  A-13,  to  a  1x5  state  vector  and  Algorithm  A 
is  exercised.  In  the  third  stage  Instead  of  augmenting  the  state  vector 
as  shown  by  Eq.  A-17,  it  is  desired  to  maintain  the  state  vector  as  a  1x5. 
This  will  be  done  using  algorithm  B  as  follows. 

We  wish  to  end  up  with  initial  conditions  for  the  third  stage  that 
is  k-l“20  in  Equations  8 *,9*  Algorithm  B.  Therefore, .take  k«21  and  write 


w^^^ (20,19)  - 


11  — I  — , 

1  (20-10)^ 

^  ^  2  3! 

3! 

0  1  1 

1  (10)^ 

2! 

3! 

0  0  11 

10 

0  0  0  1 

1 

(2U) 


(22A) 


(23A) 
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Note  x^^^[19|l9]  is  a  1x5  state  vector  and  P^^^[19|l9]  is  a  5x5  covariance 
matrix  of  x^^^.  They  are  given  from  algorithm  A  or  previous  processing. 
Then 

P(20|19)  »  w^^^(20,19)  P^^^[19|l9]  (20,19)'^ 

4x4  covariance  matrix 

K(20)  -  P(20|'19)  e^  [Pj^j^(20|l9)  +  R^(20)3"^ 

P(20l20)  »  [I^-K(20)ejl  P(20|l9)  [I^(20)eJ]'''^ 

+  K(20)  R  (20)  K(20)'’'^ 

x(20|20)  -x^^^  120119]  +K(20)|y2Q  -  x^^^^  [20ll9]  }  (27A) 

Note  that  the  *  is  deleted  from  the  estimate  x  since  once  the  collapse 
form  of  the  state  vector  is  used,  it  is  no  longer  a  minimum  variance  esti- 
mate. 

We  have  now  obtained  initial  conditions  given  by  equations  26A  and 
27A  in  terms  of  the  4x1  state  vector.  In  the  next  stage  (U=3)  we  augment 
the  state  vector  to  accommodate  the  impulse  input  b2^  and  use  algorithm  A 
beginning  with  equation  !'■  in  the  modified  foim  shown  in  Equation  22 


(24A) 

(25A) 

(26A) 
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w|^^(k,k-l)*  = 

1  1 

h  i 

2!  3! 

1 

3! 

(k-20)^ 

0  1 

1  1, 

1 

21 

(k-20)2 

0  0 

1  1 

1 

1! 

(k-20)^ 

0  0 

0  1 

(k-20)^ 

« 

0  0 

0  0 

(k-20) 

M^^^(k)  -  ej 

P^^^[20,20]*  = 

’P(20,20) 

o" 

0 

2 

‘"2_ 

(20,20)  = 

’x('’n,20) 

E{b.,  }  ■=  0 
2n 

(28A) 


(29A) 


(30A) 


(31A) 


Equations  28A  to  31A  are  now  the  input  to  algorithm  A  starting  with 
equation  5*  and  k=i>21.  Note  that  at  k=30  when  the  next  Impulse  input  occurs 
one  may  loop  back  to  equation  1'  of  algorithm  A  and  let  the  order  of  the 


filter  grow  or  collapse  the  state  vector  to  dimension  1x4  at  observation 
k-30  keeping  the  state  vector  at  cimension  5  during  the  next  stage.  We 


note  that  once  the  order  of  the  filter  has  been  reduced,  the  state  estimates 
are  no  longer  minimum  variance.  They  are  still  unbiased.  This  is  the  price 
to  be  paid  to  save  the  storage  and  processing  of  the  larger  dimension  state 


vector. 


APPENDIX  B 


ADAPTIVE  VARIABLE  ORDER  FILTERS 

The  formalism  of  the  algorithms  as  presented  in  this  paper  are  suf¬ 
ficiently  simple  and  compact  to  make  possible: 

(a)  Adaptive  placement  of  knots  based  on  measured  bias  error. 

(b)  Search  procedures  to  determine  the  existence  of  a  staging  or 
maneuver  and  modification  of  the  last  knot  to  accommodate  a 
detected  maneuver. 

The  techniques  for  solving  (a)  are  readily  extended  to  the  application 
described  by  (b)  by  requiring  more  elaborate  detection  schemes.  In  the 
maneuvering  case  one  may  wish  to  determine  in  steps  of  increasing  complexity; 

(a)  If  a  state  vector  change  has  caused  a  build  up  of  bias  error; 

(b)  In  what  interval  and  how  large  a  change  occurred; 

(c)  At  what  sample  and  how  large  a  change  occurred; 

(d)  At  what  point  in  time  and  how  large  a  change  occu’-red. 

The  structure  of  the  Adaptive  Variable  Order  Filter  is  shown  in 
Figure  1.  The  observations  input  to  the  variable  order 

filter  from  which  a  predicted  state  vector  xCk+ljk)  is  obtained.  The  pre¬ 
dicted  observation  is  compared  with  the  actual  observation  and  the 

A 

residual  "  yCk+ljk)}  is  used  as  the  input  to  the  bias  detector.  The 

form  of  the  bias  detector  will  vary  with  the  application.  The  detector  may 
also  operate  on  a  block  of  s  residuals  rather  than  just  1.  The  output  of  the 
bias  detector  will  be,  an  estimate  of  the  magnitude  of  the  bias  error,  and 


possibly  positional  information  for  knot  point  determination.  Based  on 
this  output  a  decision  is  made  as  to  whether  or  not  to  change  t'.e  structure 
of  the  filter.  If  a  change  is  decided »  one  may  place  an  additional  knot. 
Increasing  the  dimension  of  the  state  vector.  This  procedure  would  be 
followed  if  continued  improvement  In  accuracy  is  attainable  and  desired. 
Alternately  one  may  place  a  knot  but  not  increase  the  dimensions  of  the 
state  vector.  This  would  be  equivalent  to  fixing  the  memory  length  of  the 
filter.  The  latter  procedure  will  require  less  computer  storage  and  may  be 
desirable  if  maximum  accuracy  is  not  required  or  Improved  accuracy  is  not 
obtainable. 
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STRUCTURE 


